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SUMMARY 


This  study  examines  factors  relevant  to  the  development  of  organizational  measures  of 
effectiveness  (MOE)  for  the  Naval  Aviation  Engineering  Service  Unit  (NAESU).  It  was 
commissioned  by  the  Commanding  Officer  of  NAESU. 

In  conducting  the  study,  we  had  to  consider  the  broad  range  of  NAESU  activities 
including:  training  personnel;  evaluating  systems,  equipment  and.  publications;  recommending 
equipment  and  maintenance  improvement;  identifying  training  needs;  and  providing  technical 
assistance  to  the  operating  forces.  Discussions  with  tech  reps  led  to  the  development  of  a  list  of 
nineteen  products  and  services  provided  by  NAESU.  Discussions  with  various  NAESU 
customers  showed  they  had  different  opinions  regarding  NAESU  functions  in  each  of  these  areas. 
To  deal  with  these  differing  perceptions  we  found  it  useful  to  divide  NAESU  customers  into  four 
categories:  clients,  senior  level  users,  “I”  level  users,  and  “O”  level  users.  We  then  surveyed  these 
four  broad  groupings  and  asked  participants  to  evaluate  the  nineteen  products  and  services 
provided  by  NAESU.  An  analysis  of  the  services  and  responses  led  us  to  conclude  that  it  would 
be  useful  to  cluster  the  nineteen  services  into  four  broad  categories  of  training,  advice, 
maintenance  and  liaison.  We  were  then  able  to  do  some  data  analysis  that  provides  a  clearer 
picture  of  what  customers  expect  from  NAESU.  This  was  necessary  before  a  meaningful 
discussion  of  MOE’s  could  be  conducted. 

Using  a  GAO  report  as  a  guide,  we  concluded  that  measures  of  economy  and  measures  of 
effectiveness,  as  defined  in  the  report,  could  be  realistically  applied  to  NAESU.  Measures  of 
productivity  however,  are  not  applicable  to  NAESU.  Calculation  of  productivity  measures  such 
as  quantity  (input/output  ratios),  quality  (according  to  standards),  and  cost  (unit  cost  of  output)  are 
simply  not  feasible  measures  of  NAESU  activities.  The  input  varies  and  is  not  predictable  either 
as  to  quantity  or  degree  of  difficulty;  output  is  not  defined  in  quantitative  measures  and  must 
reflect  qualitative  differences  in  the  individual  problems  presented  to  NAESU  for  solution. 
Standards  for  the  type  of  work  NAESU  does  in  the  problem  solving  area  do  not  exist;  standards 
for  teaching  performance  relate  more  to  school  house  environments  than  to  the  on-the-job  training 
provided  by  NAESU  tech  reps.  Unit  cost  of  output  can  not  be  calculated  if  the  units  of  output 
can  not  be  defined. 
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Measures  of  effectiveness  involving  client  satisfaction,  economic  impact  and  contribution 
to  objectives  can  be  addressed,  however.  We  recommend  that  customers  and  tech  reps  be  sampled 
on  a  regular  basis  for  an  evaluation  of  NAESU  performance.  The  results  should  be  maintained  in 
a  database  that  would  in  itself  become  a  valuable  tool,  indicating  trends  and  changes  in 
performance  and  perhaps  providing  advance  warning  of  developing  problems.  Such  an  approach 
requires  a  more  sophisticated  understanding  of  effectiveness  measures  than  that  required  when 
using  measures  of  productivity. 

The  report  concludes  with  six  recommendations  that  focus  on  two  roles  of  NAESU’ s 
organizational  components.  These  roles  include  assuring  the  continuing  competence  of  NAESU 
tech  reps  and  creating  internal  and  external  structures  that  enable  tech  reps  to  provide  needed 
products  and  services. 
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DEFINING  ORGANIZATIONAL  MEASURES  FOR  NAESU 

INTRODUCTION 

Specialists,  who  may  be  DOD  civilians,  military  personnel,  or  private  industry  contractors, 
provide  engineering  and  technical  services  (ETS)  to  military  technicians.  The  function  of  the 
specialists  is  to  provide  an  on-call  level  of  expertise  above  and  beyond  that  available  from  within 
the  military  units.  The  Naval  Aviation  Engineering  Service  Unit,  NAESU,  provides  ETS  to  the 
Navy  and  Marine  Corp  air  communities.  It  does  so  through  Detachments,  assignments,  or  tech 
assists  to  various  unit  locations.  How  to  measure  the  success  of  NAESU  in  performing  this 
function  is  a  question  that  both  NAESU  and  NAVAIR  wish  to  address. 

PURPOSE 

This  study  was  commissioned  by  the  Commanding  Officer  of  NAESU.  As  budgetary 
resources  became  more  scarce,  he  wanted  to  develop  and  define  organizational  measures  that 
justified  the  need  for  the  NAESU  products  and  services  and  that  indicated  actions  to  be  taken  to 
achieve  greater  operational  efficiency  and  customer  satisfaction.  Additionally,  the  Government 
Performance  Review  Act  of  1993  mandated  that  government  activities  develop  and  use 
performance  measures  and  implement  benchmarking  as  standard  management  practices.  This 
study  is  an  effort  to  comply  with  the  act  and  to  develop  relevant,  useful  measures. 

BACKGROUND 

The  military  departments  operate  under  a  philosophy  that  military  personnel  should  be 
able  to  operate  and  provide  basic  maintenance  for  all  equipment.  The  departments  design 
technical  training,  skill  development,  and  personnel,  logistics,  and  management  systems  toward 
this  end.  Under  this  philosophy,  civilians  or  contractors  perform  depot  maintenance.  "O" 
(operations)  level  maintenance  and  "I"  (intermediate)  level  maintenance,  if  it  exists,  are  the 
responsibility  of  military  technicians. 

The  basic  philosophy  is  to  provide  initial  and  advanced  technical  training  in  service 
schools  and  then  put  the  newly  trained  military  technicians  to  work  in  their  assigned  fields.  The 
basic  school  graduate  can  be  considered  an  apprentice.  He  or  she  will  become  a  journeyman 
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through  a  combination  of  experience,  on-the-job  training,  mentoring  by  more  experienced  military 
personnel,  and  attending  specialty  schools.  For  this  process  to  be  effective  requires  that  all 
elements  of  the  logistics  functions  work  at  full  efficiency.  Due  to  the  random  nature  of  some  of 
the  processes,  military  technicians  seldom  are  able  to  attain  or  maintain  a  sufficiently  high  level  of 
competence  without  the  assistance  of  engineering  and  technical  services.  DOD  civilian,  military 
or  contractor  persomiel,  known  as  ETS  or  tech  reps,  provide  these  services.  They  assist  military 
technicians  in  the  installation,  modification,  operation,  and  maintenance  of  equipment  and 
systems.  The  services  generally  take  the  form  of  advice  and  training,  although  direct  performance 
of  maintenance  may  be  perfonned  occasionally.  The  tech  rep  involvement  in  these  inter¬ 
relationships  is  illustrated  in  Figure  1. 


The  Need  for  TECHREPS 
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The  REAL  WORLD  is  full  of  PROBLEMS; 

_ TECH  REPS  enable  us  to  overcome  some  of  them. 


Figure  1. 


Although  the  primary  purpose  of  Engineering  and  Technical  Services  is  the  provision  of 
training,  tech  reps  perform  a  broad  range  of  activities  that  include:  training  persomiel,  evaluating 
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systems,  equipment  and  publications,  recommending  improvements,  identifying  training  needs, 
and  providing  technical  assists  to  the  operating  forces.  The  Federal  Acquisition  Regulation 
(FAR)  includes  Engineering  and  Technical  Service  as  a  sub-set  of  Part  37.2  -  Advisory  and 
Assistance  Services.  Additionally,  NAESU  provides  contract  and  technical  oversight  of  Foreign 
Military  Sales  (FMS)  Contractor  Engineering  Technical  Services  (GETS)  and  Contractor 
Maintenance  Services  (CMS).  The  performance  of  NAESU  in  these  areas  is  not  a  part  of  this 
study. 

The  Defense  Business  Operating  Fund  Handbook  (replaced  by  Working  Capital  Funds) 
stated  that  “performance  measures  are  quantitative  expressions  of  the  success  of  achieving  a 
specific  objective. . .  Successful  perfonnance  means  that  the  provider’s  products  or  services  must: 
be  what  the  customer  needs;  be  available  when  the  customer  requirement  exists;  and,  consume  the 
least  amount  of  limited  DOD  resources. . .  Providers  are  required  to  describe  their  perfonnance 
measures  for  the  categories  of  timeliness,  quality,  and  customer  satisfaction. . .  Timeliness 
measures  are  typically  expressed  as  average  days  it  takes  for  a  customer  to  receive  a  product  or 
services. . .  Quality  measures. .  .are  typically  expressed  as  the  number  of  defects  in  the  product. . . 
or  number  of  customer  complaints  recorded.  Customer  satisfaction  is  a  measure  of  how  well 
provider  output  confomis  to  customer  expectations. .  .(it)  is  best  measured  through  direct  contact 
with  customers.” 

Technical  representative  services  are  by  their  nature  qualitative.  The  development  of 
measures  of  effectiveness  is  doomed  if  one  seeks  only  production-oriented  measures.  The  issues 
of  timeliness  and  customer  satisfaction  seem  to  be  the  most  important  considerations  in  evaluating 
the  effectiveness  of  tech  rep  organizations.  Recognizing  the  difficulty  in  developing  traditional  ' 
industrial  measures,  we  chose  to  focus  on  what  it  is  that  tech  reps  provide  to  an  organization. 
Analyzing  what  they  provide  and  what  customers  want  would  help  us  address  the  issues  of 
timeliness  and  customer  satisfaction  as  indicators  of  NAESU  perfomiance.  In  perfomiing  this 
study  we  were  interested  in  organizational  perfonnance  measures  for  NAESU  Headquarters, 
Detachments,  and  Regions,  not  the  perfomiance  of  individual  tech  reps. 
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STUDY  APPROACH 


Performance  measurement  is  commonly  instituted  to  serve  as  the  basis  for  strengthening 
accountability  or  for  resource  allocation.  The  perfonnance  measurement  system  should  be 
embedded  into  the  ongoing  management  process  so  the  measures  can  be  used  in  strategic 
planning,  operational  planning,  operational  control,  program  evaluation,  and  manager/personnel 
perfomiance  evaluation. 

The  use  of  performance  measures  varies  by  level  in  the  organization.  At  the  outcome  level, 
strategic  or  executive  level  measures  should  provide  infomiation  on  the  achievement  of  strategic 
goals,  overall  level  of  customer  satisfaction,  and  outcome  evaluation.  At  the  output  level,  middle 
management  measures  should  provide  infomiation  on  the  stability  of  the  organization  processes, 
contribution  to  the  strategic  goals,  areas  of  resource  shortage,  and  value  added  by  various 
activities  and  ou^uts.  Finally,  at  the  process  level,  operational  measures  should  provide  infomia¬ 
tion  on  the  balance  of  organization  processes  and  quality  control.  At  all  levels,  areas  to  be 
examined  include  customer  satisfaction,  employee  satisfaction  and  productivity. 

The  development  of  perfomiance  measures  for  an  organization  must  respond  to  several 
considerations:  What  characteristics  of  the  organization’s  activities  are  to  be  measured?  What 
kinds  of  perfomiance  indicators  should  be  used?  How  are  the  measures  to  be  obtained?  What  is 
the  priority  of  the  different  activities? 

In  general,  classes  of  productivity/perfomiance  indicators  for  organizations  are  concerned 
with  the  following:  efficiency,  effectiveness,  timeliness,  productivity,  quality,  innovation,  quality 
of  work  life,  and  financial  performance. 

In  our  opinion,  efficiency  and  productivity  are  not  applicable  to  tech  rep  activities  and 
financial  performance  and  quality  of  work  life  are  of  questionable  relevance.  Efficiency  is 
measured  as  a  unit  of  output  per  unit  of  time,  money,  or  other  resource;  productivity  is  typically 
measured  as  standard  output  per  unit  of  time.  ETS  provides  a  differential  output  in  response  to 
variable  situations  and  therefore  efficiency  and  productivity  would  not  be  valid  perfomiance 
indicators.  Quality  of  work  life  is  the  result  of  a  large  number  of  factors,  including  aspects  of  the 
work  environment  that  are  not  under  NAESU  control.  However,  some  measure  of  job  satisfaction 
or  work  environment  should  be  included.  Financial  performance  is  typical  of  private,  not  public 
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sector,  organizations.  It  could  apply  to  the  cost-savings  aspects  of  some  tech  rep  activities,  such 
as  fixing  items  that  would  otherwise  incur  the  large  costs  of  depot  repair,  salvaging  materials  or 
other  activities  that  result  in  significant  future  cost  reduction.  This  however,  is  difficult  to  apply. 
Even  when  the  savings  far  exceed  the  cost  of  the  tech  rep,  the  savings  are  not  in  the  same  budget 
categories  and  are  not  managed  by  the  same  organization.  NAESU  manages  the  tech  rep  costs, 
but  the  customer  manages  the  savings. 

For  performance  measures  to  be  most  meaningful,  they  should  be  crafted  to  fit  the 
environment  in  which  the  organization  operates.  In  characterizing  ETS,  key  issues  include  the 
level  and  sources  of  uncertainty  in  their  environment,  such  as: 

•  Customer-related  uncertainties  tied  to  the  number  and  qualifications  of  the  personnel 
assigned  to  a  unit,  e.g.: 

-Is  the  unit  staffed  with  the  correct  number  and  rank  of  personnel?  A  typical 
squadron  is  generally  undemianned  and  has  30%  or  mote  turnover  in  its  personnel 
each  year,  providing  an  atmosphere  of  continual  change. 

-How  qualified  are  the  junior  military  technicians?  Have  they  received  the 
schooling  appropriate  to  the  position?  More  extensive  training  of  first-term 
enlistees  prior  to  unit  assignment  may  not  be  cost-effective;  about  60%  of  them 
will  be  gone  by  the  end  of  their  enlistment.  Have  they  had  the  opportunity  to  use 
their  training  before  it  is  forgotten?  Newly  assigned  personnel  are  often  diverted 
for  months  to  other  duties  before  assuming  their  assigned  technical  position, 
seriously  reducing  the  carryover  from  training  to  job. 

-Are  the  senior  personnel  experienced  in  the  system  and  able  to  teach  and  assist  the 
junior  personnel?  Where  closed-loop  Detailing  is  not  used,  senior  enlisted 
personnel  may  not  be  experienced  in  the  currently  assigned  systems  and  may  be 
unable  to  provide  expertise  for  the  junior  technicians.  To  be  promoted,  personnel 
must  have  experience  at  both  “O”  and  “I”  levels.  Because  these  jobs  are  vastly 
different,  the  rotation  reduces  the  ability  of  the  senior  to  mentor  the  junior. 

•  Task-related  uncertainties  tied  to  the  equipment,  e.g.: 

-Are  forthcoming  weapon  system  changes  or  modifications  beyond  the  experience 
and  skills  of  assigned  operating  and  maintenance  personnel?  Lacking  funds  for 
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major  procurements,  the  services  are  making  continuous  incremental  changes  in 
equipment  the  norm  for  technological  modernization. 

-Are  forthcoming  test  equipment  or  procedure  changes  beyond  the  experience  or 
skills  of  assigned  personnel? 

-How  radical  and  pervasive  are  the  changes?  The  greater  the  changes,  the  greater 
the  need  for  tech  rep  services. 

Additionally,  how  the  customer  organization  balances  the  conflicting  objectives  of 
capability  improvement  (training  function)  and  product/process  operation  (maintenance  function) 
greatly  affects  what  is  expected  of  tech  reps.  Organizations  also  vary  in  terms  of  emphasizing 
long-term  or  short-term  improvement  in  the  organization’s  capability.  There  appears  to  be  a  bias 
in  favor  of  the  short-term,  this  week  or  next,  in  order  to  process  current  work  and  meet  flight  hour 
and  up-time  expectations.  It  is  likely  that  few  people  in  power  in  the  squadrons  and  air  stations 
are  very  concerned  about  the  long-term,  next  year  and  beyond.  A  number  of  users  expressed 
concern  and  interest  in  the  medium-term,  an  upcoming  work-up  or  deployment,  for  example. 

This  focus  arises  because  the  organization  will  be  placed  in  a  situation  where  it  is  almost  totally 
reliant  on  its  internal  personnel,  because  ETS  generally  do  not  deploy. 

In  the  private  sector,  the  customer  is  typically  both  the  receiver  and  purchaser  of  a  good  or 
service.  In  the  public  sector,  the  receiver  and  purchaser  (provider  of  funds)  are  typically  different 
persons  or  organizations.  This  requires  that  the  "customer"  be  more  carefully  defined.  For 
purposes  of  this  research,  we  will  define  two  sets  of  customers. 

The  first  customer  is  the  user,  defined  as  the  person  (including  an  organizational 
representative)  who  actually  receives  the  ETS  products  or  services.  Examples  include  pilots  and 
military  technicians  receiving  training,  squadron  and  wing  commanding  officers  and  maintenance 
officers,  and  CFAs  or  schools  that  receive  reports.  As  used  in  this  report,  a  user  is  a  customer  of 
NAESU  who  receives  specific  products  or  services  from  one  or  more  tech  reps.  Products  and 
services  include,  but  are  not  limited  to,  training,  advice,  reports,  recommendations,  repair, 
documents,  or  parts. 

The  second  customer  is  the  client,  defined  as  an  organization  or  person  who  provides  the 
resources,  principally  money  and  manpower  positions,  to  obtain  ETS.  Examples  include 
NAVAIR,  PMAs,  APMLs,  NAWC,  ASO,  depots,  and  type  or  fleet  commanders.  Specifically,  we 
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define  a  client  as  a  customer  of  NAESU  that  approves  ETS  requests,  justifies  funding  requests, 
allocates  funds  or  positions  to  NAESU,  or  assigns  funding  priorities  to  ETS  requests. 

STUDY  METHODOLOGY 

We  assume  that  tech  reps,  clients  and  users  know  what  they  want  from  NAESU,  and  that 
this  information  would  point  to  performance  measures.  In  an  earlier  project  (Boynton,  1995),  we 
collected  a  list  of  79  work  activities  that  ETS  said  they  performed.  Because  there  was  much 
duplication  showing  up  under  different  names,  we  asked  participants  at  the  1994  NAESU  OIC 
conference  to  review  the  79  activities.  Working  in  small  groups  of  about  six  people,  they 
discussed  and  clustered  the  activities.  We  used  their  clusters  to  develop  a  new  list  of  28  ETS 
activities.  We  then  asked  ETS  and  GETS  at  NAS  Lemoore  and  NAS  Brunswick  to  work  on 
those  groupings  and  were  able  to  reduce  the  list  to  19  activities.  This  sorting  lists  the  products 
and  services  that  ETS  provide  and  excludes  activities  that,  although  necessary,  do  not  directly 
produce  products  or  services.  These  different  lists  are  included  in  the  report  appendix. 

We  then  used  a  combination  of  interviews  and  questionnaires  with  users  and  clients  to 
elicit  information  regarding  the  19  ETS  activities,  services  or  products  they  were  receiving  from 
NAESU.  By  comparing  what  customers  expect  and  what  ETS  are  actually  doing,  we  hoped  to 
infer  the  types  of  measures  most  pertinent  to  NAESU  Headquarters,  Regions  and  Detachments 
(DETs)  as  providers  of  ETS.  Participants  were  asked  to  rate  the  importance  of  each  service  or 
product  and  then  to  grade  the  quality  of  the  service  or  product  they  received.  For  particularly 
high  or  low  marks  we  asked  for  the  criteria  used  to  make  the  quality  evaluation.  The  survey  form 
is  included  in  the  appendix. 

We  also  observed  and  participated  in  some  NAESU  on-site  ETS  reviews.  Since  requests 
for  the  assignment  of  ETS  exceed  the  positions  and  funds  available,  reviews  are  held  each  spring 
to  develop  and  assign  priorities  to  a  list  of  desired  ETS  requests  or  tasks  for  the  next  fiscal  year. 

SAMPLE  CHARACTERISTICS 

Our  customer  survey  responses  include  88  persons.  There  are  eighteen  civilian  and 
military  clients,  primarily  NAESU  program  managers  and  NAVAIR  APMLs.  Analysis  of  the 
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responses  of  the  70  users  revealed  apparent  differences  in  their  views.  Based  on  these  differences, 
we  have  defined  three  classes  of  users  -  senior  users,  level  users,  and  “O”  level  users. 

There  were  14  senior  users,  mostly  maintenance  officers  and  maintenance  master  chiefs 
E9  to  05,  from  NAS  Lemoore,  NAS  Miramar,  and  NAS  North  Island.  There  were  42  “I”  level 
users  from  NAS  Lemoore,  and  NAS  Miramar.  These  were  principally  E3  to  E9,  in  AE,  AT,  and 
AM  ratings,  but  included  four  officers.  The  “O”  level  users  included  14  respondents  from  NAS 
Lemoore,  principally  E4  to  E9,  in  AE  and  AM  ratings. 

The  sample  was  designed  to  be  broadly  representative  of  NAESU  customers.  We  were 
unable  to  attain  a  fully  representative  sample.  Additional  respondents  from  all  categories  and 
more  diverse  locations  are  needed  to  increase  the  ability  to  generalize  the  data. 
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RESULTS 


Some  of  the  information  developed  in  the  previous  study  (Boynton,  1995)  is  relevant  to 
and  included  in  this  study.  We  commented  that  perfomiance  measures  of  tech  rep  organizations 
might  encompass  areas  not  typically  thought  of  as  tech  rep  activities,  but  which  added  to  the 
overall  value  to  DOD.  One  example  was  the  Army  Logistics  Assistance  Program,  which  reported 
a  cost  avoidance  of  nearly  $85  million  in  1994.  Army  tech  reps  regularly  visited  local  units  of  the 
Defense  Reuse  Organization  and  reclaiiiied  items  that  they  knew,  through  experience  and 
networking,  could  be  used  by  their  own  or  other  organizations.  In  the  same  vein,  we  commented 
that  lack  of  experience  among  military  technicians,  base  closures,  unit  relocations,  and  changes  in 
mission  equipment  cause  much  turbulence  that  increases  the  likelihood  of  significant  costs  unless 
tech  reps  are  present. 

Tech  reps  also  reported  proposing  maintenance  procedure  changes  that  resulted  in  very 
large  savings  in  dollars,  technician  time,  and  equipment.  Even  though  the  primary  responsibility 
of  tech  reps  is  training,  these  other  activities  contribute  greatly  to  their  overall  value.  Perfomiance 
measures  should  take  cognizance  of  these  seemingly  peripheral  activities. 

Customers  and  tech  reps  cited  many  instances  of  gaps  in  the  logistics  systems  that 
generate  the  requirement  for  tech  rep  involvement.  Operational  requirements  commonly  take 
precedence  over  training  and  development  activities.  Modifications  are  made  to  equipment 
without  full  training  and  documentation  being  provided  with  the  modification.  Senior  personnel 
are  not  familiar  with  the  equipment.  Basic  schools,  advanced  schools  and  introduction  of  new 
equipment  are  not  synchronized.  Equipment  is  not  designed  for  ease  of  operation  and  mainte¬ 
nance  or  modification.  Military  units  lack  a  full  complement  of  qualified  personnel.  Travel 
funds  are  not  available  to  send  personnel  to  advanced  service  or  factory  training.  Built-in  test 
equipment  fails  to  provide  appropriate  infomiation  to  persomiel.  These  gaps  are  so  serious  that 
Colon  (1994),  in  his  Naval  Postgraduate  School  thesis,  recommended  ETS  be  added  as  the 
eleventh  ILS  element. 

The  intimate  equipment  and  process  knowledge  and  continuing  contact  with  operations 
and  maintenance  place  the  tech  rep  in  a  unique  position  to  develop  recommendations  for 
improvement.  Malcolm  (1995),  in  his  Naval  Postgraduate  School  thesis,  developed  a  procedure 
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for  Determining  the  tangible  cost  savings  through  improvements  to  the  reliability  and 
maintainability  of  aircraft  weapon  systems  that  resulted  from  NAESU  technical  reports.  His 
analysis  showed  that  some  technical  reports  precipitated  fleet-wide  changes  that  resulted  in 
significant  cost  savings.  He  recommended  development  of  a  computer  program  to  track  the 
effects  of  NAESU  technical  reports  on  a  continuing  basis. 

Our  interviews,  with  personnel  at  all  levels,  provided  examples  of  significant  cost 
avoidance  as  a  direct  result  of  tech  rep  actions  on  a  local  level.  Cost-avoidance  is  very  attractive 
as  a  measure  of  NAESU  and  tech  rep  value.  To  be  able  to  cite  examples  of  cost-avoidance  that 
far  exceed  the  annual  cost  of  maintaining  a  tech  rep  is  a  strong  incentive.  It  has,  however,  serious 
deficiencies.  Improvements  in  the  maintenance  procedures  used  by  military  technicians  have 
resulted  in  savings  of  money,  time,  and  parts.  One  NAS  estimated  it  would  cost  $47,000,  per 
quarter,  to  bring  instructors  to  the  station  to  perform  the  classroom  training  provided  by  the 
NAESU  DET.  Such  cost-avoidance,  however,  is  not  a  consistently  reliable  measure  of  NAESU 
performance.  It  assumes  that  the  same  training  would  have  been  conducted  by  some  other  means. 
Since  funds  are  seldom  available  to  conduct  all  desirable  training  or  to  send  technicians  to 
schools  or  factory  training,  it  is  likely  that  much  training  would  not  have  happened.  The  real 
value  is  the  increase  in  capability  of  the  military  technicians  as  a  result  of  training  provided  by 
NAESU.  That  capability  increase,  however,  cannot  be  evaluated  in  dollars. 

During  interviews,  tech  reps  cited  instances  in  which  they  were  able  to  make  significant 
savings  by  repairing  items  locally  rather  than  sending  them  to  depots.  Items  are  normally  sent  to 
the  depot  when  it  is  beyond  the  capability  of  the  organization  to  repair  the  item  (BCM).  The 
standard  cost,  which  may  be  several  thousand  dollars,  is  incurred  whether  the  item  requires  major 
repair,  minor  repair  or  was  misdiagnosed  and  needs  no  repair.  The  tech  rep  may  provide  a  correct 
diagnosis  or  on-site  repair,  avoiding  the  time  and  cost  of  sending  equipment  to  the  depot.  The  cost 
avoided  frees  the  funds  for  the  purchase  of  spare  parts,  fuel  or  other  uses  and  does  not  usually 
show  up  as  a  savings.  In  many  cases,  there  is  no  clear  way  to  determine  whether  the  problem 
might  have  been  solved  without  the  intervention  of  the  tech  rep.  The  tech  rep  is  in  the  best 
position  to  make  such  determination,  but  is  also  the  one  who  receives  credit  for  the  cost-avoidance. 
It  is  therefore  difficult  to  use  such  a  measure  on  a  regular  basis. 

Some  cost-avoidance  situations  are  much  more  clear-cut.  At  one  Naval  Air  Station,  a  tech 
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rep  determined  that  the  test  benches  were  not  properly  grounded.  This  resulted  in  good  items 
testing  as  faulty  and  significant  depot  costs  to  repair  and  calibrate  the  test  benches.  The 
unnecessary  cost  and  delay  were  eliminated  when  the  grounding  was  corrected.  Depot  costs  were 
subsequently  reduced  by  over  $23,000  per  month.  Comparable  savings  occurred  at  other  Naval 
Air  Stations  when  similar  problems  were  corrected. 

Due  to  extensive  knowledge  and  experience,  a  tech  rep  may  determine  that  an  engine 
problem  can  be  repaired  in  place.  This  avoids  the  cost  of  dropping  and  replacing  the  engine  and 
returns  the  aircraft  to  full  flight  status  more  quickly.  In  the  process  the  tech  rep  also  instructs  the 
military  technicians,  bolstering  their  knowledge  and  reducing  future  incidents.  The  increase  in  the 
technicians’  skill  and  knowledge  then  represents  cost  savings  in  the  future,  but  are  not  readily  and 
confidently  measured. 

When  asked,  "'What  measures  indicate  the  level  of performance  you  are  attaining  on  the 
job?  ”  NAESU  tech  rep  responses  included: 

•  The  attitude  of  the  military  technicians  and  others  they  work  with.  This  encompasses 
feedback,  the  apparent  opinion  of  the  low  level  technicians,  rapport,  instant  recognition 
and  credibility  of  the  NAESU  symbol,  and  a  feeling  that  the  customer  is  satisfied. 

•  Specific  comments  and  thanks  received,  including  tech  assist  reports  letters  of  apprecia¬ 
tion,  and  customer  comment  sheets. 

•  The  fact  that  sailors  and  marines  seek  out  the  tech  reps  and  request  repeat  visits. 

•  The  apparent  increase  in  the  military  technicians’  proficiency.  How  they  are  perform¬ 
ing,  their  ability  to  solve  problems  after  training,  and  not  receiving  calls  when  a  unit  is 
on  deployment,  are  examples  of  this. 

•  Some  aspects  of  time  enter  here.  One  could  measure  how  long  it  takes  military 
technicians  to  solve  a  problem  before  and  after  training,  the  number  of  hours  spent  on 
various  tasks,  and  the  spectrum  of  daily  activities. 

Their  replies  to  the  question  “How  do  your  activities  add  value  to  your  organization 's 
output  of  goods  and  services?  ”  included: 

•  Outcomes,  such  as  an  increase  in  the  number  of  Full  Mission  Capable  (FMC)  aircraft,  a 
decrease  in  partially  mission  capable  aircraft,  better  trained  sailors,  repair  equipment 
and  test  benches  that  work  and  stay  up,  regularly  providing  input  for  safety  of  flight 
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decision,  and  readiness  increases,  are  indicative  of  value  added. 

•  Resource  savings,  including  reducing  depot  repair  requirements  and  not  having  to  send 
military  technicians  away  to  school.  Cost  savings  can  also  result  from  development  of 
new  test  and  maintenance  procedures,  as  well  as  obtaining  usable  equipment  from  the 
Defense  Reuse  Organization. 

•  Reassuring  officers  by  providing  a  second  qualified  opinion  legitimizing  the  work  of 
the  military  technicians.  Making  the  technicians’  work  easier.  Avoiding  downtime  due 
to  knowledge  of  recurrent  problems  from  other  units. 

While  conducting  the  ETS  discussions,  we  began  interviewing  a  small  number  of  cus¬ 
tomers.  We  posed  questions  that  paralleled  those  asked  of  ETS.  Customers  viewed  tech  reps  as 
another  resource  to  get  aircraft  and  equipment  fully  mission  capable  in  the  shortest  time  possible 
and  to  keep  them  that  way.  Readiness  was  viewed  as  the  real  value  of  NAESU  and  tech  reps. 

Tech  reps  affect  readiness  by  providing  training  to  military  technicians,  by  developing 
maintenance  procedures,  by  clarifying  manuals,  and  occasionally  by  direct  repair  of  a  problem. 
Other  things  also  affect  readiness  and  may  include  the  quality  of  training  provided  by  the  service 
schools,  the  availability  of  spare  parts,  up-to-date  equipment  documentation  and  test  equipment. 
Such  things  as  organization  manning,  the  extent  to  which  personiiel  are  experienced  and  qualified 
for  the  assigned  equipment,  the  reliability  and  maintainability  of  the  equipment,  and  the  amount  of 
inspections  and  other  duties  that  take  people  away  from  operating  and  maintenance  also  impact 
readiness.  There  is  no  way  to  apportion  the  tech  rep  contribution  to  readiness  in  isolation  from  the 
other  factors. 

One  way  to  measure  the  readiness  impact  of  tech  reps  might  be  to  conduct  a  controlled 
experiment,  allowing  some  squadrons  to  use  tech  reps  but  not  others.  It  is  unlikely  that  squadron 
commanders  would  agree  to  participate  in  such  a  study,  since  the  readiness  numbers  are  a  major 
factor  in  their  perfonnance  evaluations  and  thus  their  promotion  possibilities.  Readiness  numbers 
are  also  subject  to  a  certain  amount  of  manipulation  and  many  of  our  interviewees  consider  the 
reported  readiness  figures  to  be  unreliable.  While  readiness  is  clearly  the  outcome  sought,  it  can 
not  be  reliably  measured  and  has  few,  if  any,  acceptable  proxies. 

As  measures  of  the  quality  of  ETS  perfomiance,  the  tech  reps  suggested  report  cards 
evaluating  ETS  timeliness,  responsiveness  to  requests,  approachability  and  professional 
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appearance. 

Graph  1  displays  average  customer  expectations  taken  from  the  questionnaire  we  asked 
clients  and  users  to  complete.  Under  the  category  of  “Service  Expected”  (from  NAESU),  we  asked 
them  to  use  ratings  of  “Primary”,  “Secondary/Incidental”,  or  “None”.  These  were  later  coded  as  3, 
2  and  1  respectively  and  used  to  calculate  the  average  expectancies.  Clients  rated  “Provide  OJT” 
as  the  service  they  most  expected.  “Conduct  off-site  tech  assists”,  “Advise  on  safety/maintenance 
concerns”  and  “Provide  classroom  training”  were  the  next  three  most  highly  rated.  Users  rated 
“Identify  fleet/systemic  problems”  highest;  with  “Provide  feedback  to  training  community”, 
“Develop  improved  maintenance  techniques”  and  “Advise  on  safety/maintenance  concerns”  as  the 
next  highest,  “Conduct  off-site  tech  assists”  and  “Provide  OJT”  were  also  highly  rated. 

Graph  1  indicates  that  clients  and  all  categories  of  users  are  largely  in  agreement  regarding 
their  expectations  from  ETS.  Although  there  are  exceptions,  a  product  or  service  rated  high  or  low 
by  one  group  is  also  rated  high  or  low  by  the  other  groups.  The  exceptions  raise  some  interesting 
questions  about  the  primary  expectations  from  ETS.  For  example,  clients  and  senior  users  place 
high  expectations  on  the  provision  of  OJT  and  classroom  training,  while  O  and  I  level  users  have 
lower  expectations.  The  statistical  significance  of  these  differences  cannot  be  tested  due  to 
sampling  and  scaling  problems. 

In  order  to  examine  the  extent  of  overall  agreement,  the  average  expectations  were 
converted  to  ranks.  Table  1  shows  that  there  is  wide  agreement  on  some  services  and  products,  but 
not  on  all.  On  OJT  and  classroom  training,  clients  and  senior  users  provide  high  ranks.  “O”  and 
“I”  level  groups,  however,  rate  them  in  the  bottom  half  of  the  activities.  How  closely  the  customer 
groups  agree  may  be  indicated  by  the  correlation  of  the  four  sets  of  ranks. 

Table  2  provides  the  correlation  coefficients  for  the  rank  orders  of  the  four  customer 
categories,  clients,  senior  users,  “I”  level  users,  and  “O”  level  users.  There  is  much  higher 
agreement,  r  =  .902,  between  the  clients  and  senior  users  than  the  other  groups.  This  helps  to 
confirm  what  was  noted  in  Graph  1 . 
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CUSTOMER  SERVICE  EXPECTATIONS 


Table  1. 

Rank  Order  of  Service  Expectations  by  Category  of  Respondent 


Type  Of  Service 

Client 

Senior 

Level 

User 

“0” 

Level 

User 

fi6J55 

Level 

User 

1.  Provide  OJT 

1 

1 

9 

11 

2.  Provide  Classroom  Training 

4 

3 

12 

11 

3.  Advise  On  Safety/Maintenance  Concerns 

2 

3 

3 

7 

4.  Advise  On  Maintenance  WorkrArounds 

11 

14 

1 

4 

5.  Participate  In  ILS/TPF  Reviews 

12 

11 

17 

10 

6.  Participate  In  Pre-Design  Reviews 

17 

15 

15 

16 

7.  Participate  In  E.I.  Investigations 

14 

16 

11 

9 

8 .  Provide  Emergency  hands-on  Repair 

14 

7 

16 

9 .  Conduct  Offsite  T ech  Assists 

3 

2 

7 

4 

10.  Provide  Liaison  And  Coordination 

10 

4 

6 

1 1.  Report  On  Equipment  Status 

17 

16 

18 

18 

12.  Provide  Supply/Logistics  Assistance 

16 

19 

15 

13.  UpdateA^erify  Tech  Pubs  And  Data 

7 

11 

14 

8 

14.  Design/Build  Peculiar  Test  Equipment 

19 

16 

19 

18 

15.  Recommend  Mods/Improvements 

5 

7 

5 

11 

16.  Provide  Feedback  To  Training  Commiinity 

7 

6 

8 

1 

17.  Advise  on  Condition  Based  Maintenance 

13 

13 

13 

18.  Develop  Improved  Maintenance  Techniques 

7 

7 

6 

3 

19.  Identify  Fleet/Systemic  Problems 

5 

5 

2 

1 

Table  2 

Pearson  Correlation  of  ranks  of  expectations  by  customer  type 


Customers 

Client 

Senior  Level 
User 

“r  Level 
User 

"0”  Level 
User 

Client 

1.000 

Senior  Level 

0.902 

1.000 

‘T’ Level 

0.653 

0.562 

1.000 

“0”  Level 

0.658 

0.511 

0.765 

1.000 

The  O  and  I  level  technicians  are  the  most  directly  affected  by  the  principal  tech  rep  actions.  The 
lower  correlation  coefficients  indicate  that  the  views  of  clients  and  senior  users  are  not  fully 
representative  of  the  military  technicians  concerns.  This  implies  a  need  to  include  all  four 
customer  types  in  future  measurements. 
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DISCUSSION  AND  RECOMMENDATIONS 


One  of  the  major  aims  of  the  previous  project  was  to  enroll  customers  and  ETS  in  the  task 
of  finding  measures  of  perfomrance  for  ETS  and  their  organizations.  An  earlier  NAESU 
Customer  Service  Improvement  survey  attempted  to  measure  some  of  the  important  areas  of 
service  provided  and  quality  level  achieved.  The  perceived  quality  was  quite  high,  with  99%  of  a 
broad  spectrum  of  customers  agreeing  with  the  statement  that  "ETS  services  received  within  the 
last  year  were  satisfactory."  Development  of  perfomrance  measures  to  support  that  perception, 
however,  has  proved  to  be  a  significant  challenge  to  both  practitioners  and  academics.  We 
discovered  no  particularly  useful  measurement  methods  currently  in  use,  either  by  NAESU 
Detachments  or  other  organizations  providing  similar  services. 

At  one  point,  NAESU  established  a  task-hour  reporting  system  to  provide  data  on  what 
tech  reps  actually  do  on  a  continuing  basis.  The  task-hour  reporting  system  did  not  provide  useful 
data  and  has  since  been  abolished.  It  should  be  noted  that  the  amount  of  time  spent  on  an  activity 
may  not  be  indicative  of  its  importance. 

In  this  study,  we  used  a  list  of  tech  rep  products  and  services  and  asked  various  groups  of 
users  and  tech  reps  to  cluster  them.  They  were  given  no  guidance  as  to  the  number  or  types  of 
clusters  to  be  used,  but  simply  asked  to  group  together  those  activities  that  seem  to  go  together. 
Consistent  clusters  would  indicate  a  natural  grouping  of  activities  that  could  fomi  the  basis  for 
aggregation  of  individual  measures  that  would  facilitate  development  of  proxy  measures  for  tech 
rep  perfomrance.  Consistency  of  clusters  under  these  circumstances  was  too  much  to  hope  for. 

To  generalize,  the  principal  clusters  of  products  and  services  are  Training,  Liaison,  Advice,  and 
Maintenance.  These  may  provide  a  basis  for  NAESU  to  survey  users  regarding  the  overall 
perfomrance  quality  of  each  DET. 

A  recent  book  by  Sveiby  discusses  intangible  assets.  “Knowledge  is  a  key  intangible  asset 
and  the  ability  to  transfer  knowledge  from  one  person  to  another  is  a  key  business  capacity.”  In 
dealing  with  intangible  assets,  there  are  no  specific  tasks  to  measure.  This  characterizes  the  work 
of  ETS.  Management  of  intangible  assets  can  be  considered  as  having  three  aspects,  the 
competence  of  the  employees,  the  internal  structures  (e.g.,  systems  and  processes),  and  the 
external  stmctures  (e.g.,  customer  relations,  supplier  relations,  and  organization  image). 


19 


Quality  is  ultimately  defined  by  the  customer’s  perception.  Tech  reps,  as  the  principal 
providers  of  NAESU  goods  and  services,  must  do  so  in  a  manner  that  meets  the  customers’ 
expectations.  This  implies  delivering  the  right  services  at  the  right  time.  The  role  of  the  NAESU 
organizational  components  then  is  to  assure  the  continuing  competence  of  ETS  and  to  provide  the 
internal  and  external  structures  that  enable  tech  reps  to  provide  those  products  and  services. 

•  Is  NAESU  responsive  to  customers’  needs,  wants,  wishes,  and  requests? 

•  Does  it  research  customer  desires,  anticipate  them,  and  plan  ways  to  meet  them? 

•  Does  NAESU  provide  opportunities  and  resources  for  its  tech  reps  to  develop  their  individual 
competence  and  teamwork? 

•  Does  NAESU  facilitate  the  interactions  of  tech  reps  and  their  customers  and  suppliers? 

•  Does  the  image  of  NAESU  positively  affect  the  tech  reps  work? 

GAO  report  GGD-98-137,  although  developed  for  Grant  Program  design  features, 
provides  a  model  of  aspects  of  perfomiance  that  seem  appropriate  to  a  service  organization  such 
as  NAESU.  Figure  2,  taken  from  the  report,  is  based  on  “0MB  documents  prepared  to  assist 
agencies  in  meeting  . . .  performance  measurement  requirements.” 

It  identifies  four  aspects  of  performance  -  inputs,  activities,  outputs,  and  outcomes  -  each 
representing  a  major  step  in  the  process  of  converting  program  resources  into  program  results. 
Referring  to  Figure  2,  it  would  seem  that  measures  of  economy  and  measures  of  effectiveness 
could  be  developed  for  NAESU.  Measures  of  productivity,  however,  would  be  very  difficult  and 
arbitrary.  Production  units  of  input  and  output,  required  to  develop  input/output  ratios,  are  not 
definable  in  this  kind  of  intangible,  knowledge-intensive  environment.  Standards  do  not  exist  for 
the  type  of  problem  solving  work  perfomied  by  NAESU  and  teaching  standards  apply  more  to 
schoolhouse  environments,  with  a  set  clientele  and  curriculum,  than  to  on-the-job  and  ad  hoc 
situations  in  which  much  NAESU  training  is  conducted.  On  the  other  hand,  client  reach, 
customer  satisfaction,  economic  impact  and  contribution  to  objectives  can  be  addressed. 

RECOMMENDATION  1.  We  recommend  that  NAESU  infomi  NAVAIR  that  the 
measures  of  productivity  are  not  meaningful  for  NAESU.  It  should  request  that  NAVAIR  exempt 
it  fi'om  productivity  measures. 
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Figure  2:  Performance  Spectrum  and  Measurement 
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Modified  and  adapted  from  GAO  report  GGD-98-137 


RECOMMENDATION  2.  We  recommend  that  a  pilot  study  be  undertaken  in  FY  1999  to 
develop  and  validate  a  survey  instrument  for  future  use. 

RECOMMENDATION  3.  We  recommend  that  NAESU  take  random,  unbiased  samples 
of  the  customer  population.  The  sample  should  be  taken  at  least  every  six  months  and  should 
focus  on  the  quality  of  NAESU  service  in  the  four  broad  categories  of  training,  liaison,  advice  and 
maintenance.  The  sample  should  include  the  four  categories  of  NAESU  customers,  i.e.  clients, 
senior  users,  “I”  level  users  and  “O”  level  users.  The  survey  should  look  for  a  few  simple 
indicators  that  can  be  aggregated  to  address  the  quality  of  service  provided  by  NAESU  by  DET, 
Region,  and  overall.  Over  time,  the  survey  should  lead  to  the  development  of  a  data  base  that 
facilitates  identification  of  problems  and  provides  an  indication  of  whether  the  problems  are  short 
term  (one  time  or  temporary),  long  term  or  possibly  systemic  changes  in  NAESU  effectiveness. 

RECOMMENDATION  4.  We  recommend  that  the  survey  include  the  areas  of  timeliness, 
responsiveness  to  requests,  approachability,  customer  reach,  and  customer  satisfaction.  It  should 
include  opportunity  for  open-ended  responses  for  cost-avoidance  and  procedural 
recommendations. 

RECOMMENDATION  5.  We  recommend  that  NAESU  address  customer  and  task 
uncertainties  by  continuing  the  annual  on-site  ETS  reviews. 

RECOMMENDATION  6.  We  recommend  that  a  separate  sample  survey  of  tech  reps  be 
taken  every  six  months.  It  should  include  their  job  satisfaction,  work  enviromnent,  developmental 
needs  and  organizational  support. 
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APPENDIX:  LISTS  AND  FORMS 


The  five  exhibits  attached  to  this  appendix  trace  the  development  of  the  lists  of  ETS 
activities. 

79  ETS  ACTIVITIES 

An  initial  list  of  17  activities  was  developed  firom  conversations  with  tech  reps.  This  list 
was  used  in  a  questionnaire  for  tech  reps  and  customers.  They  were  requested  to  add  any  other 
activities  they  felt  to  be  important.  Customers  added  26  items  and  tech  reps  provided  36  more. 
These  were  added  to  the  original  17  without  editing  for  duplication  or  relevance. 

ACTIVITY  CLUSTERS 

The  79  activities  were  sorted  into  groups  or  clusters  by  six  groups  of  NAESU  ETS  and 
DET  OICs.  The  clusters  were  named  by  the  groups. 

28  ETS  ACTIVITIES 

Elimination  of  duplications,  very  minor  activities,  and  those  that  were  not  tech  rep 
activities  resulted  in  a  list  of  28  activities. 

19  ETS  ACTIVITIES 

This  list  resulted  from  consolidation  of  activities  and  removal  of  those  that  did  not  provide 
an  output  good  or  service. 

ETS  SURVEY  FORM 

The  19  activities  were  used  in  this  questionnaire.  It  is  the  basis  for  evaluating  customers 
expectations  from  NAESU. 
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79  ETS  ACTIVITIES 


Items  1-17  are  the  activities  of  ETS/LARS  used  in  the  survey  questionnaire.  In  response  to  an  open  "other’ 
line,  activities  18-43  were  added  by  customers  and  44-79  by  ETS/LARS. 

ACTIVITIES  LISTED  ON  THE  QUESTIONNAIRE 

1.  Provide  OJT 

2.  Provide  classroom  training 

3.  Liaison/Coordination 

4.  Report  on  equipment  status 

5.  Advice  on  Personnel  or  Management 

6.  Update/verify  tech  pubs/data 

7.  Design/Build  peculiar  test  equip 

8.  ILS/TPF  Reviews 

9.  Pre-design  reviews 

10.  Investigate  Els  as  Tech  Advisor 

1 1 .  Actual  hands-on  maintenance 

12.  Supply  assistance 

13.  Recommend  mods/improvements 

14.  Administration/paperwork 

15.  Off-site  tech  assists 

16.  Organization-initiated  Education/Development  of  tech  rep's  own  skills 

17.  Self-initiated  Education/Development  of  tech  rep's  own  skills 

ACTIVITIES  ADDED  BY  CUSTOMERS 

1 8 .  Maintenance  direction/advice 

19.  Course  development 

20.  Continuity 

21.  Networking 

22.  In-depth  knowledge  and  experience 

23.  Equipment  repair  continuity 

24.  Systems  history  knowledge 

25.  Ability  to  provide  AF-level  systems  support 
26-  Corporate  knowledge 

27.  Provide  assistance  to  "O"  level  customers 

28.  Advice  on  safety  related  items/maintenance  concerns 

29.  Advice  on  work-arounds  to  maintenance 

30.  Advice  on  trouble  shooting  accuracy 

31.  Advice  and  concerns 

32.  Training  raw  reciuits 

33.  Trouble  shootirrg  beyond,  or  not  covered  by,  tech  manuals 

34.  Provide  feedback  to  training  conurruuity 

35.  Resident  expert 

36.  Continuity 

37.  Depot  engineering  contacts 

38.  Company  contacts 

39.  Enthusiasm  to  accomplish  missions 

40.  Importance  of  prior  military  service 

41 .  Proficiency  of  machinery 

42.  BCnowledge  of  systems 

43.  Recommendatiorrs  for  Condition-Based  maintenance 


25 


ACTIVITIES  ADDED  BY  ETS/LARS 


44.  NG/USAR  liaison  visits 

45.  Safety  of  equipment' 

46.  Evaluate  unit/system  readiness 

47.  Supply  assistance 

48.  New  equipment  fielding 

49.  Assisting  with  supply  problems 

50.  Very  instrumental  in  getting  this  program  restarted 

5 1 .  Represent  unit  at  product  improvement  groups  and  conferences 

52.  Devise  non-typical  repairs  in  wartime  scenario 

53.  Technical  assistance 

54.  Daily  help 

55.  Supervision  of  other  employees  (75%  to  25%) 

56.  Equipment  verification 

57.  CSRA/reviews/inspections 

58.  Coordination  of  FITWING/NAESU  training  (IWSR,  advanced  training,  etc) 

59.  Education  of  the  military  into  the  potential  of  having  personnel  trained  to  provide  training  and  continuity 

60.  Helping  units  solve  problems 

61.  Vendor  liaison 

62.  Mobilization  (Guard  and  reserve) 

63.  Revalidation  (Guard  and  reserve) 

64 .  Logistics/tra nsport/tracking 

65.  Acquiring  needed  hardware  from  CECOM 

66.  Maintenance  technique  improvement 

67.  Local  equipment  logistics  assistance 

68.  Subordinate  work-skill  improvement 
•69.  Detachment  administration  interface 

70.  Providing  technical  guidance  to  other  personnel 

71.  Identifying  fleet  problems/systems 

72.  Verify  and  feedback  report  PMS 

73.  Tech  assist  air  travel 

74.  Meetings  (PMR) 

75.  Factory  training 

76.  Maintain  "MAMS"  spare  parts  for  trouble  shooting  and  fault  isolation 

77.  Other  tasks  besides  technical 

78.  Liaison  with  other  groups  on  EW 

79.  Working  military  operations 
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ACTIVITY  CLUSTERS 


Participants  in  the  August  1994  NAESU  OIC  conference  were  divided  into  6  groups  (about  6 
persoiis/group)  and  asked  to  group  or  cluster  the  79  ETS  activities.  They  were  then  asked  to  name  their  clusters  as 
they  reported  back  to  the  whole  group. 


•  GROUP  1 

1.  Tech  Assists:  53  15  22  54  70  71  76  79  27  1 1  60  33  35  18 

2.  Training:  2  19  42  66  32  75  1  58  59 

3.  Field  Engineering  Feedback:  7  46  51  58  52  56  441  72  1013  45  6 

4.  Advice:  31  30  29  28  5  43 

5.  Supply:  12  64  65  67  47 

6.  Liaison:  3  36  61  44  49  20  34  21  78 

7.  Reviews:  8  37  38  26  74  57  40  9 

8.  Administration:  14  73  55  68  17  16  77  69 

9.  Reserves:  63  62 
Throw  Away:  25  50  39  23 

•  GROUP  2 

1.  Training:  32  1  66  75  19  59  68  2  58  34  11  17  16 

2.  Supply  assistance:  49  47  64  67  12  76  65 

3.  Knowledge/Tech  Assist:  22  42  35  41  24  27  71  73  15  4  33  43  52  40 

4.  Liaison/Continuity/Design  Review:  36  45  25  50  69  78  9  56  6  7  48  8  51  13  57  72  14 

5.  Advice:  29  18  70  30  53  28  54  10  31  60  5  46 
Throw  away:  55  37  62  63  79  77 


•  GROUP  3 

1.  Training:  2  19  32  68  22  24  42  35  16  75 

2.  Tech  Assist:  11  15  27  1  18  30  33  28  43  52  53  54  59  60  66  70  71  7  10  23  29 

3.  Logistics:  47  49  64  65  67  76 

4.  Administration:  5  14  55  69  77 

5.  Feedback/Reporting:  38  37  3  44  61  20  26  17  21  74  23 

6.  Conference/review:  8  9  51  57  6  4  41  45  46  48  13  56  72  34 

7.  Liaison:  3  20  26  37  38  44  50  58  61  74  78 
Trash:  63  40  39  79  73  12  36  25  62  31 

•  GROUP  4 

1.  Liaison:  3  4  18  28  29  31  34  21  37  38  43  44  12  47  49  61  65  69  78  36 

2.  Maintenance:  11  23  41  52  76 

3.  Quality  Assurance:  6  8  10  45  64  67  71  72  74  4  7  9  13  14  46  48  51  56  57 

4.  Training:  2  1  15  16  17  19  25  27  30  32  53  54  58  59  60  66  68  70  75  77  39 

5.  Experience:  20  22  24  26  33  35  40  42 

6.  Other:  55  73  79  62  63  50 
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•  GROUP  5 

1.  Provide  OJT:  1  15  27  1 1  29  32  52  54 

2.  Provide  classroom  training:  2  19  58  59  66  68  70  75 

3.  Liaison/Coordination:  1831  53738  44  506178 

4.  Proficiency  of  Machinery:  45  23  56  28 

5.  Resident  Expert:  20  21  24  22  26  33  40 

6.  Supply  Assistance:  49  64  65  67  76  4 

7.  Administration/Paperwork:  55  69  17  39  16 

8.  Recommend  Mods/Improvements:  34  10  72  25 

9.  Technical  Assistance:  60  71  73  79  62  63 

10.  Other  Tasks  besides  Technical:  7  43  46 

1 1 .  ILS/TPF  reviews:  •  6  9  48  5 1  57  74 

•  GROUP  6 

1.  Training:  45  41  60  54  59  53  33  32  30  29  28  70  68  27  25  22  19  18  2  1  15 

2.  Liaison:  8  57  61  64  67  72  3  65  12  9  13  21  47  49  58  74  34  5  31  78  44  43  71  46  51  37  38 

3.  Administration:  77  73  69  14  55 

4.  Research:  6  10  48  56  4  66  52  7 

5.  Hands-on  Maintenance:  76  11 

6.  Personal  Management/Improvement:  75  17  16 

7.  Continuity  of  Corporate  Knowledge:  42  40  36  35  26  24  23  20 
Unknown:  39  50  79  63  62 
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28  ETS  ACTIVITIES 


1.  Provide  OJT 

2.  Provide  classroom  training 

3.  Liaison/coordination 

4.  Report  on  equipment  status 

5.  Advice  on  persoimel  or  management 

6.  Update/verify  tech  pubs/data 

7.  Design/build  peculiar  test  equip 

8.  ILS/TPF  reviews 

9.  Pre-design  reviews 

10.  Investigate  Els  as  Tech  Advisor 

1 1 .  Actual  hands-on  maintenance 

12.  Supply/logistics  assistance 

13.  Recommend  mods/improvements 

14.  Administration/paperwork 

15.  Off-site  tech  assists 

16.  Organization-initiated  Education/Development  of  tech  rep’s  own  skills 

17.  Self-initiated  Education/Development  of  tech  rep's  own  skills 

18.  Maintenance  direction/advice 

19.  Advice  on  safety  related  items/maintenance  concerns 

20.  Advice  on  work-arounds  to  maintenance 

21.  Advice  on  trouble  shooting  accuracy 

22.  Trouble  shooting  beyond,  or  not  covered  by,  tech  manuals 

23.  Provide  feedback  to  training  community 

24.  Recommendations  for  Condition-Based  maintenance 

25.  Maintenance  technique  iihprovement 

26.  Providing  technical  guidance  to  other  persomiel 

27.  Identifying  fleet/systemic  problems 

28.  Working  military  operations 
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19  NAESU  ETS  ACTIVITIES 


1 .  Provide  OJT 

2.  Provide  classroom  training 

3.  Advise  on  safety/maintenance  concerns 

4.  Advise  on  maintenance  work-arounds 

5.  Participate  in  ILS/TPF  reviews 

6.  Participate  in  pre-design  reviews 

7.  Participate  in  El  Investigations 

8.  Provide  actual  hands-on  maintenance 

9.  Conduct  off-site  tech  assists 

10.  Provide  liaison/coordination 

1 1 .  Report  on  equipment  status 

1 2.  Provide  supply/logistics  assistance 

1 3.  Update/verify  tech  pubs/data 

14.  Design/build  peculiar  test  equip 

15.  Recommend  mods/improvements 

16.  Provide  feedback  to  training  community 

17.  Advise  on  Condition-Based  maintenance 

18.  Develop  improved  maintenance  techniques 

1 9.  Identify  fleet/systemic  problems 
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